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(57) Abstract: A method for digital content adaptive watermarking robust 
against general affine transforms, cropping and compression is disclosed. 
The method is based on a wavelet domain additive watermarking with a mul- 
tiresolution perceptual mask (1 1) determined by the stochastic noise visibil- 
ity function NYF of the cover image x. It is shown how to encode messages 
b and how to design the periodic watermark w in order to recover, based on 
the watermark Fourier magnitude spectrum F(w), general affine transform 
and compression attacks. Furthermore, it is demonstrated that the method 
is flexible and compatible with any message encoding technique and in par- 
ticular with turbo codes, BJCR-, log-MAP and max-log-MAP decoders and 
with low-density parity check codes. 
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Method for Adaptive Digital Watermarking Robust 
Against Geometric Transforms 



TECHNICAL FIELD 

The invention refers to the field of digital watermarking 
5 and in particular to generating and extracting digital 
watermarks for images or video sequences. 

BACKGROUND ART 

Two major conflicting constraints on digital image water- 
marks are invisibility, i.e. avoiding perceptible arti- 
10 facts in the watermarked or stego image, and robustness, 
i.e. resistance against various intentional or uninten- 
tional attacks such as affine geometric transforms (rota- 
tion, scaling, aspect ratio changes, shear) , translation, 
cropping, image compression etc. 

15 In earlier solutions the information to be embedded was 
encoded using e.g. M-ary modulation (M. Kutter, "Perfor- 
mance Improvement of Spread- Spectrum based Image Water- 
marking Schemes through M-ary Modulation" , Lecture Notes 
in Computer Science: Third International Workshop on In- 

20 formation Hiding, Springer, Vol. 1768, 237-252) or alge- 
braic error correction codes (ECC) (J. R. Hernandez, 
F. Perez-Gonzalez, J. M. Rodrigez and G. Nieto, "The im- 
pact of channel coding on the performance of spatial wa- 
termarking for copyright protection", Proc . ICASSP'98, 

25 2973-2976, May 1998) . M-ary encoding suffers from a high 
complexity of the watermark demodulator, whereas error 
correction codes are less effective. On the other hand, 
turbo codes and BCJR, log-MAP or max-log-MAP decoders 
(C. Berrou and A. Glavieux, "Near optimum error correc- 

30 ting coding and decoding: turbo-codes", IEEE Trans. 
Comm., 1261-1271, October 1996) or low-density parity 
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check codes (R. Gallager, "Low- density parity-check 
codes", IRE Transactions on Information Theory, January 
1962) have not been applied to digital watermarking. 

The perceptual mask has to determine the optimal level of 
5 allowable distortions for the watermark embedding. An 
overview of empirical masking methods based on the deter- 
ministic models of the human visual system (HVS) is given 
by S. Voloshynovskiy, A. Herrigel, N. Baumgartner and 
T. Pun, "A Stochastic Approach to Content Adaptive Digi- 

10 tal Image Watermarking" , Lecture Notes in Computer Sci- 
ence: Third International Workshop on Information Hiding, 
Springer, Vol. 1768, 211-236. The main problem consists 
in the content-adaptive watermarking, since in the most 
cases the HVS mask is given in the coordinate domain and 

15 watermark embedding is performed in some transform domain 
(block-wise and full-frame discrete Fourier (DFT) or dis- 
crete cosine (DCT) transforms, wavelet or Radon trans- 
forms) . The embedded watermark is then transformed to the 
coordinate' domain and mapped by the mask. More recent me- 

20 thods try to utilize either transform domain masking ba- 
sed on a just noticeable difference that originates from 
the image compression applications (I- Podilchuk and 
W. Zeng, "Image-Adaptive Watermarking Using Visual Mo- 
dels", IEEE Journal on Selected Areas in Communications, 

25 16(4), 525-539), or combined masking in frequency and 
coordinate domains (U. S. Pat. No. 6,031,914). In the 
latter, a major drawback is that both a frequency-domain 
and a spatial-domain perceptual mask must be applied con- 
secutively in order to achieve invisibility. Furthermore, 

3 0 the watermark can only be extracted when the unmarked 
image is accessible. 

In the above-mentioned publication by S. Voloshynovskiy 
et al . a stochastic perceptual mask based on a noise vi- 
sibility function NVF is proposed. However, since the NVF 
35 and the perceptual mask are developed only in the spatial 
coordinate domain, they are not well adapted for calcula- 
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tions in a frequency domain and are not easily modifiable 
by restrictions stemming from the frequency domain. 

Robustness against geometrical distortions has so far 
been relied on using a transform invariant domain 
5 (J. Oruanaidh and T . Pun, "Rotation, Scale and Translati- 
on Invariant Spread Spectrum Digital Image Watermarking", 
Signal Processing 66(3), 303-317, 1998), or an additional 
template (WO 96/36163), or an Autocorrelation Function 
(ACF) of the watermark itself (M. Kutter, "Watermarking 

10 resistent to translation, rotation and scaling", Proc . 
SPIE Int. Symp. on Voice, Video, and Data Communication, 
1998) . The transform invariant domain approach suffers 
from interpolation and accuracy problems, therefore re- 
quires comparatively large images of size 512x512, and 

15 cannot recover rotational and aspect ratio changes simul- 
taneously. The template approach needs a computationally 
expensive exhaustive search for recovering these trans- 
forms together, and it is susceptible to unauthorized re- 
moval of template peaks . In the ACF approach the water- 

20 mark is replicated in the image in order to create 4 
repetitions of the same watermark. The corresponding 9 
peaks in the ACF are used to recover undergone geometri- 
cal transformations. However, the descending heights of 
the ACF peaks shaped by the triangular envelope function 

25 reduce the robustness of this approach against geometri- 
cal attacks accompanied by a lossy compression. The need 
for computing two discrete Fourier transforms (DFT) of 
double image size to estimate the ACF poses problems in 
real time applications with large images. 

30 A further requirement for digital watermarking is a suf- 
ficient information capacity of the watermark. In order 
to attach a unique identifier to each buyer of an image, 
a typical watermark should be able to carry at least 
60-100 bits of information. However few publications deal 

35 with 60 or more bits. 
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From the above review it is concluded that the existing 
technologies exhibit at least one of the following pro- 
blems : 

1. Constrained spatial domain modulation for content- 
5 adaptive watermarking. 

2. Inability to resist against geometrical transforms ac- 
companied by the lossy JPEG compression. 

3. Low simultaneous robustness against lossy JPEG (DCT- 
based) and wavelet compression. 

10 4. Low robustness against printing/rescanning for high 
quality commercial magazine printing. 

5. No protection against intentional template removal. 

6. Less than 60 bits encoding for limiting the complexity 
of the watermark demodulator or decoder. 

15 DISCLOSURE OF THE INVENTION 

It is the object of the present invention to provide an 
improved method of the type mentioned above that is in 
particular capable of dealing with at least some, pre- 
ferably all of these problems. This object is achieved by 
0 the subject-matter as set forth in the independent 
claims . Preferred embodiments are described in the depen- 
dent claims. The present invention is well suited for wa- 
termarking still images and video data. 

The invention resides in a method for embedding a digital 
5 watermark w in an image x, comprising the steps of enco- 
ding a digital message b in a codeword c, mapping the 
codeword c and allocating the mapped codeword c Into a 
block B, producing a symmetric block B' of fourfold size 
by flipping and copying the block B once in every block 
0 direction, tiling the symmetric block B' in order to ge- 
nerate a symmetric digital watermark w with a period B' 
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and embedding the watermark w in the image x in order to 
obtain a stego image y. By tiling or repeating the basic 
block B' a plurality of times, periodic features are in- 
troduced into the final watermark w both in the coordina- 
5 te and frequency domain, that can be used for retrieving 
affine transform attacks undergone by the stego image. 
The block flipping makes the watermark w robust against 
stego image flipping attacks, i.e. rotations by 90°, 180° 
or 270° , and reduces the number of ambiguities during 
10 estimation of the undergone geometrical attacks. Further- 
more, the block flipping increases the invisibility of 
the watermark w by visually decorrelating its repetitive 
structure in the coordinate domain. 

Preferred embodiments are: adding a secret-key-dependent 

15 reference watermark w re f in remaining orthogonal spatial 
locations of the block B to render the resulting water- 
mark w robust against translation or cropping attacks un- 
dergone by the stego image y; up-sampling pixels of the 
block B or equivalently B' at least twofold in each block 

20 dimension for creating robustness against the finite re- 
solution of image input or output media, such as printers 
and scanners; using a turbo code or a low-density parity 
check code for encoding the digital message b thereby 
keeping the block size small; using a secret encryption 

25 key for encrypting the codeword c and/or a secret block 
allocation key for block allocation to improve the safety 
of message hiding and decoding; embedding the watermark w 
in the image x in wavelet sub-bands k,l, wherein k is a 
resolution index and 1 a direction index thereby provi- 

30 ding full compatibility of the embedding procedure with 
the recently developed wavelet-based compression standard 
JPEG2 00 0 . 

The invention further resides in a method for embedding a 
watermark w in an image x, comprising the steps of: 
35 calculating image wavelet components x k j(i,j) and watermark 
wavelet components w kl (i, j) for pixel locations i,j, based 
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on the x kl (i,j) calculating in the wavelet sub-bands k, 1 a 
noise visibility function NVF k ,i(i,;j) and therefrom a per- 
ceptual mask PMk,i(i/j) for masking the ^(i,;) and embed- 
ding the masked watermark wavelet components into the 
5 XkjQ'j) to Produce stego image wavelet components 

and calculating by an inverse discrete wavelet transfor- 
mation (IDWT) the stego image y. By using a stochastic 
approach to image analysis based on the NVF and by defi- 
ning in the wavelet domain the NVF and a NVF-based per- 
10 ceptual mask PM, invisibility constraints, frequency- 
domain constraints and geometric robustness constraints 
can be incorporated into a single perceptual mask PM. 

Preferred embodiments refer to: calculating the noise vi- 
sibility function NVF k/ i(i,j) from a stationary generali- 

15 zed Gaussian model or a non- stationary Gaussian model of 
the image x; incorporating in the perceptual mask 
PM k ,i(i,j) watermark strengths S e k ,i for edges and textures 
of the image x with a weighting factor 1-NVF and water- 
mark strengths S f k ,i for flat regions of the image x with a 

20 weighting factor NVF; wavelet-domain embedding by multi- 
plying PM k( i ( i , j ) with w k l (/, j) and adding x u (i, j) ; adapting 
the watermark strengths S\,i and/or S f k ,i in order to take 
advantage of a frequency- dependent modulation transfer 
function (MTF) and/or a spatial orientational dependence 

25 of the human visual system (HVS) ; in particular choosing 
S e k,i - S f k,i for a majority of or all wavelet sub-band in- 
dices k, 1 and/or choosing S e i,i>S e 2/ i >se 3,i >se 4,i <se 5,i and 
S f i / i>S f 2/ i >sf 3,i > S f 4,i <sf 5 / i for k=1...5 and/or choosing 
S e k/ i<S e k , 3 , S e k ,2^S e k , 3 and S f k/1 <S f k ,3, S f k/2 ^S f k/3 , wherein the 

30 indices 1=1 and 1=2 denote a horizontal and vertical ori- 
entation and 1=3 a diagonal orientation in the image x; 
and/or compressing the image x in the wavelet sub-bands 
k, 1 before the watermark embedding in order to realize 
"compressed domain watermarking". 

35 The invention further resides in a method for extracting 
a watermark w, that was previously embedded according to 
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invention, from a possibly attacked stego image y' , com- 
prising the steps of: calculating an estimated watermark 
w from the stego image y ! , detiling the estimated water- 
mark w into the symmetric block B 1 by summing correspon- 
5 ding portions of a plurality of periods of the estimated 
watermark w and converting the symmetric block B 1 into 
the block B and extracting the digital message b from the 
block B. This extraction method assures that full advan- 
tage is taken of the tiling and flipping operations per- 
10 formed during watermark embedding. 

Preferred embodiments refer to: using a maximum a poste- 
riori probability (MAP estimation) for calculating the 
estimated watermark w; estimating a watermark-covariance 
matrix R w globally; estimating an image -covariance matrix 

15 R x locally; estimating and correcting a geometric affine 
transform from peaks in the spectral power density |f(w) | 2 
and/or the autocorrelation function (ACF) w*w of the 
estimated watermark w; cross-correlating the block B with 
a reference watermark w re f to compensate translations 

20 and/or cropping; down-sampling a previously up-sampled 
block B by averaging identical neighbouring pixels; using 
secret key for block deallocation and/or message decryp- 
tion; and/or using a BJCR, a log-MAP or a max-log-MAP de- 
coder for soft decoding previously turbo-coded digital 

25 messages b. 

Other objects, features and advantages of the present in- 
vention will become apparent from the description in con- 
nection with the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

3 0 The drawings show in 

Fig. 1 an embodiment for generating a digital watermark 
w robust against geometrical transforms; 
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Fig. 2a exemplary wavelet pyramids of a cover image x 
("Lena"), in Fig. 2b of the digital watermark w, 
and in Fig. 2c of the noise visibility function 
NFV of the cover image x; 

5 Fig. 3a the modulation transfer function (MTF) of the hu- 
man visual system (HVS) and a state-of-the-art 
non-adaptive embedding ; 

Fig. 3b a 1-dimensional wavelet decomposition and Fig. 3c 
an adaptive embedding according to a preferred 
1 o embodiment ; 

Fig. 4 a 2 -dimensional wavelet decomposition related to 
the MTF; 

Fig. 5 an embodiment for embedding the digital watermark 
w robustly in the wavelet domain; 

15 Fig. 6 an embodiment for extracting and decoding the di- 
gital watermark w from an attacked stego image 
y' ; and 

Fig. 7a-7d watermark extraction using spectral power den- 
sity peaks: watermark w-cover image x-stego image 
20 (Fig. 7a) , an estimated 

Fig. 7a-7d digital watermarks w, w extracted by using 
spectral power density peaks: cover image x-stego 
image y (Fig. 7a) ; watermark estimated by denoi- 
sing a stego image y (Fig. 7b) , a compressed ste- 
25 go image y' (Fig. 7c) and a rotated and com- 

pressed stego image y' (Fig. 7d) . 

In the drawings identical parts are designated by identi- 
cal reference numerals. 
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MODES FOR CARRYING OUT THE INVENTION 

Formulation of a preferred embodiment: 

We formulate the embedding process as an additive con- 
tent-adaptive watermarking in the wavelet domain with the 
5 watermark possessing special spatial structure that en- 
ables to recover general affine transforms. We assume 
that we are given a cover image to be watermarked denoted 
x. If it is an RGB image we work with the luminance com- 
ponent, though the same methodology can be applied to 
10 other color spaces. The given message (the copyright in- 
formation or URL address) in binary form b = (b XJ .^b L J is to 
be embedded in the cover image x — (x l9 ... 9 x N ) T of size M l xM 2 , 
where N = M l -M 2 . 

Message encoding and spatial allocation: 

15 Fig. 1 shows an example of watermark creation. The mes- 
sage b is first encoded 1 in a codeword c using prefera- 
bly either low-density parity check codes (R. Gallager) 
or turbo codes (C. Berrou and A. Glavieux) , the publica- 
tions of which are herewith incorporated in this applica- 

20 tion in their entirety by reference. The maximum rate at 
which these codes can be used is known to be bounded be- 
low channel capacity. However, the existence of simple 
iterative decoding schemes and their outstanding error 
performance more than compensate this weakness. 

25 The codeword c is then mapped 2 from {0,1} to {-1,1} and 
encrypted 3 by multiplying on a key-dependent sequence p 
with following spreading 4 over a square block B of size 
N l xN l with some density D using a secret key. In the gen- 
eral case, it could also be a rectangular block B or a 

3 0 block B of any shape. 

The key-dependent reference watermark w re f is added 5 to 
the above block B in some or all remaining orthogonal 
spatial locations . The reference watermark w re f is used to 
recover cropping and translation based on the cross- 
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correlation with the attacked stego image y' . The refer- 
ence watermark w re f consists of a binary key-dependent se- 
quence {-1,1} and its length is determined by the embed- 
ding density (1-JD) as is described above. 

5 The resulting block B is up-sampled 6 by the factor 2 to 
receive a low-pass watermark and then flipped and copied 
7 once in each direction, producing a symmetric block B' 
of size 4^x47^. The flipping 7 is performed to visually 
decorrelate the structure of the repeated watermark w and 

10 to reduce the number of ambiguities during estimation of 
the undergone geometrical attacks. Finally, the 4N l x4N 1 
block B' is repeated preferably over the whole image 
size, resulting in a symmetrical and periodical watermark 
w with periods T x -T 2 = 4N 1 . In our applications we use L=64 

15 bit messages that are encoded using the turbo code 
(K=132) . The block size is chosen to be N l =19 resulting 
in a density L>= 0 . 7 4 in order to have exactly 2 times 
repetition of the codeword c in every block B. The final 
block B ' after up-sampling 6 and flipping 7 has the size 

20 76x76 u The scheme is very flexible in respect to the en- 
coding 1 and can use any known modulation technique or 
even more advanced error correction codes (ECC) . 

Stochastic multi-resolution image modeling and watermark 
embedding : 

25 The principle of watermark embedding is shown in Fig. 5. 
To embed the above obtained watermark w in a cover image 
x a linear additive scheme is used in the wavelet domain. 
Both the cover image x and the watermark w are first de- 
composed into multi-resolution sub-band pyramids using 

30 the (discrete) Forward Wavelet Transform (FWT or DWT) . 
First, the cover image x is padded to a square size of 
the nearest larger number to the original cover image 
size of power of 2 in order to apply a standard wavelet 
transform DWT, 9. In the numeric example below, N w = 5 

35 levels are used for the DWT based on the Daubechies 8-tap 
filter (M. Vetterli and J. Kovacevic, "Wavelets and Sub- 
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band Coding", Prentice Hall, 1995). This results in 6 re- 
solution sub-bands k or scales. Scales from 1 to N w =5 
are also divided into 3 components corresponding to dis- 
tinct orientations 1, for horizontal (H) , vertical (V) 
5 and diagonal (D) directions. The lowest scale k-N w +l 
however consists of only a low-pass component. Fig. 2a 
shows the pyramids of the cover image x and Fig. 2b of 
the watermark w. 

The watermarking process is applied and adapted to each 
10 (£,/) wavelet sub-band component separately as described 
below. Finally, the stego image y is reconstructed by 
computing the Inverse Wavelet Transform (IWT, 12) of the 
digitally watermarked image pyramid. 

An important issue is the adaptation of the watermark w 

15 to the properties of the HVS, i.e. content-adaptive wa- 
termarking. Assuming we are given a masking function of 
the HVS, we wish to embed the above described watermark 
into the cover image x keeping it under the threshold of 
visual imperceptibility . We propose to use a stochastic 

20 perceptual mask PMk,i(i,j), 11 based on a noise visibility 
function (NVF) proposed by Voloshynovskiy et al and ear- 
lier developed only for the coordinate domain. Here the 
NVF is for the first modified in order to include the 
multi-resolution paradigm in the stochastic framework to 

25 take into account a modulation transfer function (MTF) of 
the HVS and to match the proposed watermarking algorithm 
with the recently developed image compression standard 
JPEG2 0 00 for future integration. This practically means 
that different watermark strengths S or S e , S f are as- 

30 signed to different image sub-bands k, 1. Such a modifi- 
cation leads to a non-white spectrum of watermarks w 
being matched with the MTF. Previously this could not be 
achieved with the coordinate-domain based version of the 
NVF . The second reason to use wavelet domain embedding is 

35 motivated by the desire to incorporate the anisotropy of 
the HVS to different spatial directions in the perceptual 
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mask PMk,i (i / j ) / 11. The coordinate domain version of the 
NVF used only an isotropic image decomposition based on 
the extraction of a local mean from the original image or 
its high-pass filtered counterpart. In the wavelet domain 
5 k, 1 the image coefficients in 3 basic spatial direc- 
tions, i.e. horizontal (1=1), vertical (1=2) and diagonal 
(1=3), are received as a result- of the decomposition, 
which therefore allows to exploit the anisotropic sensi- 
tivity of the HVS . As a result, the watermark strengths S 
10 can be varied for different orientations 1 in the pro- 
posed mask PMk,i(i,j), 11. 

The NVF is based on a stationary Generalized Gaussian 
(sGG) model or on a non-stationary Gaussian model of the 
cover image x or the cover image wavelet coefficients 
15 Si:* i (*"»./) for every sub-band k, 1. Accordingly the perceptual 
edge and texture masking in the wavelet domain is modeled 
based on the NVF , of pixel (/, j) , for each sub-band compo- 
nent (kj) : 

(*, j) = - 2 , ( Gl ) 

20 (J~ kj is the global variance of the wavelet image coeffi- 
cients from sub-band (kj) , and the watermark wavelet com- 
ponents w kJ (/, 7) can be written as 



with (G2) 



and ***** (G3) 

25 where T(t) is the gamma function. The NVF ' s features for 
a given sub-band k, 1 are determined by the global sub- 
band variance a2 and by the shape para'meter y u (i,j) which 
is estimated based on the moment matching method 
(A. Jain, "Fundamentals of digital image processing", 



WO 02/13138 



PCT/IB00/01089 



13 

Prentice-Hall, 1989) . An example of the NVF pyramid for 
image "Lena" is shown in Fig. 2c. 

Finally the weighted watermark is added to the cover 
image x using the following embedding rule: 

5 y « (?. J)= & J)+ ((i - NVF kJ (i, j)} si, + NVF U (i, y> si, )• w kJ (i, j) { Q4 } 

wherein the factor in front of the Wj^foj) defines the 
perceptual mask PM k ,i(i,j). The y k j(hj) are the obtained 
stego wavelet components and PM k ,i(i,j) • w k j(i,j) are the 
perceptually masked watermark wavelet components. S\ / i is 

10 an embedding strength for the edges and textures, and S f k/ i 
is a strength for the flat regions of the cover image x. 
Visual masking is ensured first by choosing S e k ,i greater 
than S f k/ i for edges and textures hiding, and second by 
using adapted strengths for each resolution, and even for 

15 each orientation based on the properties of the MTF . An 
example of practically used embedding parameters accor- 
ding to the MTF properties, considering cover image pixel 
values in the range [0,255], are: 

0.1 0.1 0.2 0 
0.2 0.2 0.5 0 
0.5 0.5 1 0 
1120 
2 2 3 1 

where rows k denote decreasing resolutions, and columns 1 
each orientation. The watermark strengths or embedding 
parameters S\ /lf S f k ,i reflect very important particular! - 

25 ties of the HVS . First, the strengths of watermark for 
the diagonal directions are chosen to be higher than for 
the vertical and horizontal ones. This is motivated by 
the fact that the anisotropy sensitivity of the HVS to 
the diagonally oriented patterns is lower than for the 

30 vertical and horizontal directions. Therefore, it makes 
possible to embed stronger watermark signals there. More- 
over, it allows to obtain, as a result, better robustness 
against lossy compression (both JPEG-DCT and wavelet 
JPEG2 000) . The lossy compression is exploiting the same 
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property of the HVS to allocate smaller amounts of bits 
in the diagonal directions for the image coding. Therefo- 
re, the proposed embedding technique utilizes both infor- 
mation about the HVS and the quantization of lossy image 
5 coding to increase the robustness of the watermark w . 

Second, the MTF of the HVS has a typical frequency depen- 
dence, as is shown in Fig. 3a (A. Jain, p. 55) , with a 
maximum in a low frequency range and decreasing side 
lobes at very low and middle to high frequencies. In the 

10 case of non-adaptive watermark embedding (Fig. 3a) , the 
typical additive white Gaussian watermark has a uniform 
spectrum. A uniform increase of the watermark strength or 
equivalently watermark power density would violate the 
invisibility constraint at low frequencies. However, 

15 there still remains a lot of space for watermark embed- 
ding at the very low, middle and high frequencies below 
the threshold of imperceptibility . To exploit this oppor- 
tunity we use the wavelet sub-band decomposition (Fig. 
3b: wavelet subbands V1...V5 for a 1-dimensional example) , 

20 wherein the watermark strength could be adopted according 
to the local properties of the MTF (Fig. 3c) . This adap- 
tation to the MTF is reflected in the proper choice of 
the embedding parameters. S e , S f that have maxima in the 
corresponding frequency sub-bands k along each spatial 

25 direction 1 (Fig. 4) . 

Third, the particular properties of the given image x 
within each sub-band k, 1 are taken into account using 
local watermark strength control based on the NVF, as 
discussed earlier. This feature has image dependent cha- 
30 racter oppositely to the previous two properties that 
characterize the HVS in general. Therefore, the proposed 
watermark embedding technique utilizes both general fea- 
tures of the HVS as well as local statistics of cover 
images x. 
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Watermark extraction and message decoding: 

A generalized block-diagram of watermark extraction is 
shown in Fig. 6. The embedded watermark w is first esti- 
mated 13, w from the possibly attacked stego image y T . 
5 Secondly, geometric distortions which may have occurred 
are retrieved and compensated 15 to obtain a rectified 
watermark w rec , by analyzing 14 the Fourier transform F(w) 
or the spectral power density magnitude | F ( w ) | 2 and/or an 
autocorrelation function (ACF) w*w of the estimated wa- 

10 termark w. The ACF is preferably obtained by w*w= 
F"" 1 ( | F (w) | 2 ) with F" 1 () being the inverse DFT . The tiled 
blocks are then detiled or averaged 16 in order to get an 
estimate of the embedded redundant sequence according to 
the maximum likelihood (ML) estimate for a Gaussian chan- 

15 nel . The cropping and translation are compensated 19 
using cross-correlation 18 with the reference key- 
dependent watermark w re f, 17. Finally, the message is de- 
crypted 2 0 and decoded 21. 

Watermark estimation: 

20 To estimate the watermark w a maximum a posteriori prob- 
ability (MAP) estimate is used: 

w = argmaxfe x (/ J w\ p w (w)} 

, (G5) 

wherein p w Q is the probability density function of the 
watermark w. Assuming that the image y' and watermark w 
25 are conditionally independent identically distributed lo- 
cally Gaussian, i.e. x~N(x,R x ) and w ~ N(Q,R W ) with the 
covariance matrices R x of the image x and R w of the wa- 
termark w, where R w also includes the effect of percep- 
tual watermark modulation, one can determine: 

30 ™ ^ * (G6) 

where the mean values Y'-x are assumed to be equal and 
where R x = max(0, R y - R w ) is the ML-estimate of the local 
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variance {R x -&ll with I=identity matrix) and R y is an es- 



timated covariance matrix of the original stego image y. 

An important issue is the estimation of the watermark co- 
variance matrix R w in the above estimate. This can be do- 
5 ne based on the available copy of the stego image y' . 
However, the severe distortions due to lossy JPEG com- 
pression could destroy the information about the texture 
masking that was used for the watermark embedding, and a 
histogram modification attack could damage the relevant 

10 information about contrast sensitivity masking. Since no 
reliable information about the perceptual mask PM is 
available after these attacks, we propose to use a global 
estimate of the watermark strength based on the available 
copy of the attacked image y' . This practically means 

15 that we assume spatial stationarity of the watermark 
R w - 6*1 . To estimate a global watermark variance we use 
the following formula: 



estimate (G7) is a global mean value of the watermark va- 
riance. Obviously, other robust versions of (G7) such as 
a robust median estimate of the variance could be 
applied, as well. 

5 Determining affine gepmetrical distortions: 

To determine the affine transformation applied to the im- 
age we compute | F ( w) | 2 from the estimated watermark w, 
where F(w) is the discrete FT. Due to the periodicity of 
the embedded information, the estimated watermark spec- 
0 trum possesses a discrete structure. Assuming that the 
watermark w is white noise within the block B, the spec- 
trum of the watermark w will additionally be uniform. 
Therefore, |F(w) | 2 shows • aligned and regularly spaced 
peaks. For a T x xT 2 -periodical watermark w, peaks will 




(G7) 



where 6y(rn 9 n) is a local variance of the stego image y in 
0 the coordinates (ra,n), for an image of size NxM . The 
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have periods M1/T1 and M 2 /T 2 for a 2-D FT domain of size 
MjXM 2 . If an affine distortion was applied to the stego 
image y, the peaks layout will be re-scaled, rotated 
and/or sheared, but alignments will be preserved. There- 
5 fore, it any affine geometrical distortion can be esti- 
mated from these peaks by fitting alignments and estimat- 
ing periods . 

Finding the matched points between the extracted posi- 
tions of local peaks in the magnitude spectrum of the es- 
10 timated watermark (zi,Z2) and the reference grid (fi,f 2 ) 
computed based on the knowledge of the embedded watermark 
period, one can estimate the linear affine transform A 
using all matched points such that the next criterion is 
minimized: 





~ f\h 


T 


Zi z 2 


T- 


« A 












Jkfk. 




_ Z k Z k _ 





(G8) 



where p{} is a negative log-likelihood function associa- 
ted with the distribution of the misaliments and k is a 
number of matched points . In the most common case, it is 
assumed that the misalignment distribution is Gaussian, 

20 and one receives a quadratic log-likelihood function 
p{}~||| 2 and the corresponding mean square error minimiza- 
tion criterion. In the more general case, the above pro- 
blem could be solved based on the theory of robust M- 
estimators, if some ambiguity about misalignment distri- 

25 bution exists. 

Fig. 7a-7d show peaks extracted from the magnitude spec- 
trum of the watermark | F ( w) | 2 . In Fig. 7a, the real embed- 
ded watermark w is shown that was calculated by subtract- 
ing y-x using the knowledge of the cover image x in a 
3 0 non-oblivious approach, whereas in Fig. 7b the Wiener 
predicted watermark w is taken. Therefore, these peaks 
can be extracted from the stego data with high fidelity 
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based on the estimated watermark w without knowledge of 
the cover image x. This important conclusion is also con- 
nected with the observation that the embedded watermark w 
is mostly allocated in the middle frequency band. This 
5 has double importance. First, low frequencies of the 
stego image y or y' are not altered considerably in order 
not to produce visible distortions. Second, the watermark 
w will resist against lossy compression that removes 
mostly high frequency components from the image y or y' . 

10 Fig. 7c-7d show peaks extracted after lossy compression, 
without (Fig. 7c) and with (Fig. 7d) geometric distorti- 
ons, here a 37° rotation of the stego image y' followed 
by a JPEG compression with a quality factor QF=50%. In 
experiments peaks could be properly extracted from JPEG 

15 compressed images with a quality factor QF up to 50. At 
the time of patent submission, no known watermarking me- 
thod is able to resist to affine transforms combined with 
such a compression. 

Recovering translation and cropping is based on the refe- 
20 rence key-dependent watermark w re f, 17 (Fig. 6) . To reduce 
computational complexity and using the information about 
the periodicity of the watermark w we first perform wa- 
termark detiling 16, i.e. coherent summation of the esti- 
mated watermark w from different periods. This results in 
25 the symmetric block B' that is converted to the final 
block B of size N l xN l . The block B is correlated 18 with 
the reference watermark w re f . The maximum of cross- 
correlation 18 makes possible to establish the undergone 
translation or cropping that is easily compensated 19. 

3 0 Message decoding: 

Assuming that attack, prediction and extraction errors 
could be modeled as additive Gaussian, the detector is 
designed -using the ML formulation for the detection of a 
known signal (projection sets p are known due to the key) 
35 in Gaussian noise, that results in a correlator detector 
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r = (w,p) 



(G9) 



In more general cases, the detector should be designed 
for stationary non-Gaussian noise or for the non- 
stationary Gaussian case. Finally, given an observation 
5 vector r , the decoder that minimizes the conditional pro- 
bability of error, assuming that all codewords b are 
equi -probable, is given by the ML decoder: 



Based on the central limit theorem (CLT) most researchers 
10 assume that the observed vector r can be accurately ap- 



noise for a large sample space. 

We use symbol -by- symbol MAP decoder for the turbo code 
that is commonly known as a BCJR decoder, a log-MAP or a 
15 max- log-MAP decoder, i.e. soft decoding, that is known to 
be superior in comparison with the hard decoding for 
Gaussian channels . 

While there are shown and described presently preferred 
embodiments of the invention, it is to be distinctly un- 
20 derstood that the invention is not limited thereto but 
may be otherwise variously embodied and practiced within 
the scope of the following claims. 




b 



(G10) 



proximated as the output of an additive Gaussian channel 



25 
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CLAIMS 

1. A method for embedding a digital watermark w in an 
image x, comprising the steps of 

a) encoding (1) a digital message b in a codeword c, 
5 b) mapping (2) the codeword c and allocating (4) the 

mapped codeword c into a block B, 

c) producing (7) a symmetric block B' by flipping and 
copying the block B once in every block direction, 

d) tiling (8) the symmetric block B' in order to ge- 
10 nerate a periodic symmetric digital watermark w 

and 

e) embedding the watermark w in the image x in order 
to obtain a stego image y. 

2. The method according to claim 1, comprising, between 
15 steps b) and c) , the step or steps of 

a) adding (5) a secret-key-dependent reference water- 
mark w ref to the block B in remaining orthogonal 
spatial locations of the block B and/or 

b) up-sampling (6) pixels of the block B at least 
20 twofold in each block dimension. 

3. The method according to one of the claims 1-2, com- 
prising the steps of 

a) using a turbo code or a low-density parity check 
code for encoding (1) the digital message b and/or 
25 b) using a secret encryption key for encrypting (3) 

the codeword c and/or a secret block allocation 
key for block allocation (4) and/or 

f) embedding the watermark w in the image x in wave- 
let sub-bands (k,l). 

30 4. A method for embedding a watermark w in an image x, 
-in particular according to one of the previous 
claims, comprising the steps of 

a) calculating by a discrete wavelet transform (DWT, 
9, 10) image wavelet components x kl (i,j) of the image 
35 x and watermark wavelet components w kJ (i,j) of the 
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watermark w, for pixel locations (i,j) and wavelet 
sub-band indices k, 1, 

b) based on the image wavelet components x k j(i>j) calcu- 
lating a noise visibility function NVF k ,i(i,j) in 

5 the wavelet sub-bands (k,l) and therefrom a per- 

ceptual mask PMk,i (i, j ) (11) for masking the water- 
mark wavelet components and 

c) embedding the masked watermark wavelet components 
into the image wavelet components x kl (i 7 j) to produce 

10 stego image wavelet components y k ,ii}^j) an< 3 calcula- 

ting by an inverse discrete wavelet transformation 
(XDWT, 12) the stego image y. 

5. The method according to claim 4, comprising the steps 
of 

15 a) calculating the noise visibility function 

NVFk,i (i, j ) from a stationary generalized Gaussian 
model or a non-stationary Gaussian model of the 
image x and/ or 
b) based on the noise visibility function NVF k ,i(i,j) 

20 calculating a perceptual mask (11) 

PM kf i(i, 3) = (l-NVF k/1 (1,3) ) • S\,i + NVF k ,i(i, j) ) • S f k/i/ 
wherein S e k ,i are watermark strengths for edges and 
textures and S f k ,i are watermark strengths for flat 
regions of the image x, and/or 
25 c) calculating the stego image wavelet components 

y k ji},j) according to an embedding rule 

6. The method according to one of the claims 4-5, com- 
prising the steps of watermark weigthing in the wave- 

30 let sub-bands (k,l) by watermark strengths S e k/ i for 

edges and textures and/ or by watermark strengths S f k ,i 
for flat regions of the image x in order to exploit a 
frequency- dependent modulation transfer function 
(MTF) and/or a spatial orientational dependence of 

35 the human visual system (HVS) . 
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7. The method according to claim 6, comprising the steps 
of 

a) choosing the watermark strengths S e k ,i for edges and 
textures larger than the watermark strengths S f ] C/ i 

5 for flat regions for a majority of or all wavelet 

sub-band indices k, 1 and/ or 

b) adapting the watermark strengths S\,i, S f k,i as a 
function of the wavelet sub-band index k in an in- 
verse relation to a modulation transfer function 

10 (MTF) of the human visual system (HVS) and/or 

c) choosing the watermark strengths S e k,i, S f k,i as a 
function of the wavelet sub -band index 1 larger 
for a diagonal orientation (1=3) than for a hori- 
zontal (1=1) or vertical (1=2) orientation. 

15 8. The method according to one of the previous claims, 
comprising the step of subjecting the image x to a 
compression scheme in wavelet sub-bands k, 1, in par- 
ticular to JPEG2000 compression, before the embedding 
of the digital watermark w. 

20 9. A method for extracting a watermark w from a possibly 
attacked stego image y' , wherein an original stego 
image y was obtained by embedding the watermark w in 
an image x according to one of the claims 1-3 and in 
particular according to one of the claims 4-7, com- 

25 prising the steps of 

a) calculating (13) an estimated watermark w from the 
stego image y' , 

b) detiling (16) the estimated watermark w into the 
symmetric block B ' by summing corresponding porti- 

30 ons of a plurality of periods of the estimated wa- 

termark w and converting the symmetric block B ! 
into the block B and 

c) extracting (2 0, 21) the digital message b from the 
block B. 
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10. The method according to claim 9, comprising the steps 
of 

a) using a maximum a posteriori probability (MAP 
estimation) for calculating the estimated water- 

5 mark w and 

b) in particular using an approximate equation 

w = K/(R W + R X ) • (y'-y 7 ) , 

wherein R w is a watermark-covariance matrix, R x is 
an estimated image-covariance matrix, and "y 7 are 
10 mean values of the stego image y ' . 

11. The method according to claim 10, comprising the 
steps of 

a) estimating a watermark- covariance matrix R w glo- 
bally by averaging local variances a y ^ 2 (m,n) of the 

15 stego image y ' over spatial coordinates (m,n) 

and/or 

b) estimating an image-covariance matrix R x locally 
from max ( 0 , R y - R w ) , wherein R y is an estimated co- 
variance matrix of the original stego image y ba- 

2 0 sed on a maximum likelihood estimate, R w is a wa- 

termark- covariance matrix and max ( ) defines a 
maximum of its arguments. 

12. The method according to one of the claims 9-11, com- 
prising the steps of 

25 a) calculating (14) a spectral power density | F ( w) | 2 

of the estimated watermark w, wherein F(w) is a 
discrete Fourier transform (DFT) , and/or calcula- 
ting an autocorrelation function (ACF) w*w of the 
estimated watermark w, 

30 b) extracting peaks from the spectral power density 

|F(w) I 2 and/or from the autocorrelation function 
(ACF) w*w, 

c) based on the peaks estimating coefficients of a 
geometric affine transform matrix A and compensa- 

35 ting (15) geometrical distortions in the estimated 



WO 02/13138 



PCT/IB00/01089 



24 

watermark w to obtain a rectified estimated water- 
mark w rec for detiling (16) and further processing. 

13. The method according to one of the claims 9-12, com- 
prising the steps of 

5 a) generating (17) a reference watermark w re f using a 

secret reference watermark key and cross- 
correlating (18) the reference watermark w re f with 
the block B for identifying and compensating in 
the block B translations and/or cropping undergone 
10 by the stego image y' and/or 

b) averaging identical neighbouring pixels in case of 
a previously up-sampled block B. 

14. The method according to one of the claims 9-13, com- 
prising the steps of 

15 a) using a secret block allocation key for extracting 

a codeword c from the block B and/ or 

b) using a secret encryption key for decrypting (2 0) 
a codeword c of the digital message b and/or 

c) in case of a digital message b having been encoded 
20 with a turbo code, using a BJCR, a log-MAP or a 

max-log-MAP decoder for soft decoding (21) the di- 
gital message b. 
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